Neuromorphic event-based slip detection and suppression in robotic grasping and manipulation

Slip detection is essential for robots to make robust grasping and fine manipulation. In this paper, a novel dynamic vision-based finger system for slip detection and suppression is proposed. We also present a baseline and feature based approach to detect object slips under illumination and vibration uncertainty. A threshold method is devised to autonomously sample noise in real-time to improve slip detection. Moreover, a fuzzy based suppression strategy using incipient slip feedback is proposed for regulating the grip force. A comprehensive experimental study of our proposed approaches under uncertainty and system for high-performance precision manipulation are presented. We also propose a slip metric to evaluate such performance quantitatively. Results indicate that the system can effectively detect incipient slip events at a sampling rate of 2kHz ($\Delta t = 500\mu s$) and suppress them before a gross slip occurs. The event-based approach holds promises to high precision manipulation task requirement in industrial manufacturing and household services.Comment: 18 pages, 14 figure

Dynamic-vision-based force measurements using convolutional recurrent neural networks

In this paper, a novel dynamic Vision-Based Measurement method is proposed to measure contact force independent of the object sizes. A neuromorphic camera (Dynamic Vision Sensor) is utilizused to observe intensity changes within the silicone membrane where the object is in contact. Three deep Long Short-Term Memory neural networks combined with convolutional layers are developed and implemented to estimate the contact force from intensity changes over time. Thirty-five experiments are conducted using three objects with different sizes to validate the proposed approach. We demonstrate that the networks with memory gates are robust against variable contact sizes as the networks learn object sizes in the early stage of a grasp. Moreover, spatial and temporal features enable the sensor to estimate the contact force every 10 ms accurately. The results are promising with Mean Squared Error of less than 0.1 N for grasping and holding contact force using leave-one-out cross-validation method

Neuromorphic vision based contact-level classification in robotic grasping applications

In recent years, robotic sorting is widely used in the industry, which is driven by necessity and opportunity. In this paper, a novel neuromorphic vision-based tactile sensing approach for robotic sorting application is proposed. This approach has low latency and low power consumption when compared to conventional vision-based tactile sensing techniques. Two Machine Learning (ML) methods, namely, Support Vector Machine (SVM) and Dynamic Time Warping-K Nearest Neighbor (DTW-KNN), are developed to classify material hardness, object size, and grasping force. An Event-Based Object Grasping (EBOG) experimental setup is developed to acquire datasets, where 243 experiments are produced to train the proposed classifiers. Based on predictions of the classifiers, objects can be automatically sorted. If the prediction accuracy is below a certain threshold, the gripper re-adjusts and re-grasps until reaching a proper grasp. The proposed ML method achieves good prediction accuracy, which shows the effectiveness and the applicability of the proposed approach. The experimental results show that the developed SVM model outperforms the DTW-KNN model in term of accuracy and efficiency for real time contact-level classification

A survey of single and multi-UAV aerial manipulation

Aerial manipulation has direct application prospects in environment, construction, forestry, agriculture, search, and rescue. It can be used to pick and place objects and hence can be used for transportation of goods. Aerial manipulation can be used to perform operations in environments inaccessible or unsafe for human workers. This paper is a survey of recent research in aerial manipulation. The aerial manipulation research has diverse aspects, which include the designing of aerial manipulation platforms, manipulators, grippers, the control of aerial platform and manipulators, the interaction of aerial manipulator with the environment, through forces and torque. In particular, the review paper presents the survey of the airborne platforms that can be used for aerial manipulation including the new aerial platforms with aerial manipulation capability. We also classified the aerial grippers and aerial manipulators based on their designs and characteristics. The recent contributions regarding the control of the aerial manipulator platform is also discussed. The environment interaction of aerial manipulators is also surveyed which includes, different strategies used for end-effectors interaction with the environment, application of force, application of torque and visual servoing. A recent and growing interest of researchers about the multi-UAV collaborative aerial manipulation was also noticed and hence different strategies for collaborative aerial manipulation are also surveyed, discussed and critically analyzed. Some key challenges regarding outdoor aerial manipulation and energy constraints in aerial manipulation are also discussed

Energy distribution in dual-UAV collaborative transportation through load sharing

In this paper, a novel dual-UAV collaborative aerial transport strategy based on energy distribution and load sharing is proposed. This paper presents the first experimental demonstration of dual-UAV collaborative aerial transport while distributing power consumption. The demonstration is performed while distributing the power consumption between two drones sharing a load based on their battery state of charge. A numerical model of the dual-hex-rotor-payload is used to validate the proposed strategy. Numerical and hardware tests were conducted to demonstrate the load distribution using multiple UAV with certain spatial configurations. Finally, collaborative aerial transport test scenarios are performed numerically and experimentally. The simulation and experimental results show the effectiveness and applicability of the proposed strategy

A double-layered elbow exoskeleton interface with 3-PRR planar parallel mechanism for axis self-alignment

Abstract Designing a mechanism for elbow self-axis alignment requires the elimination of undesirable joint motion and tissue elasticity. The novelty of this work lies in proposing a double-layered interface using a 3-PRR planar parallel mechanism as a solution to the axis alignment problem. 3-PRR planar parallel mechanisms are suitable candidates to solve this as they can span the desired workspace in a relatively compact size. In this paper, we present the modeling, design, prototyping, and validation of the double-layered elbow exoskeleton interface for axis self-alignment. The desired workspace for the self-axis alignment mechanism is specified based on the estimated maximum possible misalignment between the exoskeleton joint and the human anatomical elbow joint. Kinematic parameters of the 3-PRR planar mechanism are identified by formulating an optimization problem. The goal is to find the smallest mechanism that can span the specified workspace. The orientation angle of the mechanism’s plane addresses the frontal frustum vertex angle of the elbow’s joint, while the translational motion allows the translational offsets between the user’s elbow and the exoskeleton joint. The designed exoskeleton axis can passively rotate around the frontal plane ±15 deg and translate along the workspace 30 mm in the frontal plane. Experimental results (quantitative and qualitative) confirmed the capability of the proposed exoskeleton in addressing the complex elbow motion, user’s satisfaction, and ergonomics

UAV payload transportation via RTDP based optimized velocity profiles

This paper explores the application of a real-time dynamic programming (RTDP) algorithm to transport a payload using a multi-rotor unmanned aerial vehicle (UAV) in order to optimize journey time and energy consumption. The RTDP algorithm is developed by discretizing the journey into distance interval horizons and applying the RTDP sweep to the current horizon to get the optimal velocity decision. RTDP sweep requires the current state of the UAV to generate the next best velocity decision. To the best of the authors knowledge, this is the first time that such real-time optimization algorithm is applied to multi-rotor based transportation. The algorithm was first tested in simulations and then experiments were performed. The results show the effectiveness and applicability of the proposed algorithm

Neuromorphic eye-in-hand visual servoing

Robotic vision plays a major role in factory automation to service robot applications. However, the traditional use of frame-based camera sets a limitation on continuous visual feedback due to their low sampling rate and redundant data in real-time image processing, especially in the case of high-speed tasks. Event cameras give human-like vision capabilities such as observing the dynamic changes asynchronously at a high temporal resolution ($1\mu s$) with low latency and wide dynamic range. In this paper, we present a visual servoing method using an event camera and a switching control strategy to explore, reach and grasp to achieve a manipulation task. We devise three surface layers of active events to directly process stream of events from relative motion. A purely event based approach is adopted to extract corner features, localize them robustly using heat maps and generate virtual features for tracking and alignment. Based on the visual feedback, the motion of the robot is controlled to make the temporal upcoming event features converge to the desired event in spatio-temporal space. The controller switches its strategy based on the sequence of operation to establish a stable grasp. The event based visual servoing (EVBS) method is validated experimentally using a commercial robot manipulator in an eye-in-hand configuration. Experiments prove the effectiveness of the EBVS method to track and grasp objects of different shapes without the need for re-tuning.Comment: 8 pages, 10 figure

Design of a novel passive Binary-Controlled Variable Stiffness Joint (BpVSJ) towards passive haptic interface application

In this paper we present the design, development and experimental validation of a novel Binary-Controlled Variable Stiffness Joint (BpVSJ) towards haptic teleoperation and human interaction manipulators applications. The proposed actuator is a proof of concept of a passive revolute joint, where the working principle is based on the recruitment of series-parallel elastic elements. The novelty of the system lies in its design topology, including the capability to involve an (n) number of series-parallel elastic elements to achieve (2n) levels of stiffness, as compared to current approaches. Accordingly, the level of stiffness can be altered at any position without the need to revert to the initial equilibrium position. The BpVSJ has low energy consumption and short switching time, and is able to rotate freely at zero stiffness without limitations. Further smart features include scalability and relative compactness. This paper details the mathematical stiffness modeling of the proposed actuator mechanism, as well as the experimentally measured performance characteristics. The experimental results matched well with the physical-based modeling in terms of stiffness variation levels. Moreover, Psychophysical experiments were also conducted using (20) healthy subjects in order to evaluate the capability of the BpVSJ to display three different levels of stiffness that are cognitively realized by the users. The participants performed two tasks: a relative cognitive task and an absolute cognitive task. The results show that the BpVSJ is capable of rendering stiffness with high average relative accuracy (Relative Cognitive Task relative accuracy is 97.3%, and Absolute Cognitive Task relative accuracy is 83%)